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Chapter 1. Principles of Human-Computer Interaction - НС! 


Chapter 2. Understanding user populations, especially 
older people 


Chapter 3. User interface design tools and practice 


Chapter 4. Designing for older and disabled people 





This session will present arguments for supporting as wide as possible user 
populations. 


Not only is this an ethically important aim, but also it makes commercial sense. 


We will look at differing user populations, their needs and expectations, and how we 
can design interactive systems to support ‘universal usability’. 


The session is split into three parts, as shown in the overview. 
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НСІ DEFINITIONS AND HISTORIC EVOLUTION 





ШД to Human- -Сотрифег Interaction — 


rocampus | а „Егазглиз + 


The term HCI was adopted in mid-1980s: 


Association for Computing Machinery (ACM): "discipline concerned with the 
design, evaluation & implementation of interactive computer systems for human 
use & with the study of major phenomena surrounding them" (1992) 


Dix: “HCI is study of people, computer technology and the ways these influence 
each other. We study HCI to determine how we can make this computer 
technology more usable by people" (1998) 


Carroll: “НС! is the study and practice of usability. It is about understanding and 
creating software and other technology that people will want to use, will be able 
to use, and will find effective when used." (2002) 





Evolution of the definition of HCI given more weight to the usability concept 
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Some implied concepts: 


Human: an individual user, 
a group of users working 
together or a sequence of 
users in an organization... 


Interaction: action 
involvinga dialog with 
feedback and control 
throughout 
performing a task 


User interface: parts 


of the computer that Computer: a computer is a device that accepts 

the user contacts with information and manipulates it for some result 
based on a program (a sequence of instructions) 
on how the data is to be processed. 


Different versions: desktop, large-scale computers, 
pocket PC, embedded system, tablet, Smartphone, 
(ОТ... 
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Why we need HCI understanding and practice? 


* Nowadays, computers are cheap and widely used by non-technical 
people, with very different skills, background, needs.. 
Computer manufacturers noticed the importance of making computers 
"user-friendly" and ease to use. 
Main challenge: How to achieve "user-friendliness" in computer 
design? 


Apple co-founder Steve Jobs said in a 1983 
speech that the company's strategy is really 
simple: 


What we want to do is to put an incredibly 
great computer in a book that you can carry 
around with you and learn how to use in 20 
minutes ... and we really want to do it with a 
radio link in it so you don't have to hook up to 
anything and you're in communication with all 
of these larger databases and other 


ENIAC 1946 
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Following 3 slides are from "Top Ten Myths about Usability" presentation by Thomas S. 
Tullis 2009 (http://www.measuringux.com/UsabilityMyths/UsabilityMyths. pdf ) 


Origins of HCI in the World War | era: 


* invention of machines 
that taxed human 
sensorimotor abilities, 
e.g., interface for aircraft 
control 
even after extensive 
training, frequent (fatal) 
errors 


Darn these hooves! I hit the wrong switch again! 
Who designs these instrument panels, raccoons? 
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— К B-17 coming in for a landing, 
The B-17 “Flying Fortress ment Кеи Car down d 
flaps fully extended. 


But they would sometimes 
come in like this... 


alina пт! 
Jil ie Quit 


The problem was.... And crash land like this. 


The consequences of a bad design 
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B-17 Cockpit METELLI As a result of the work 
“= | — done by the ћитап- 
factors engineers, 
shape coded controls 
were introduced into 


cockpits 
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Some historical evolution of the interaction with computers: 


1940s 


From the first 
mouse by D. 
Engelbart in 
1964.... 


batch computing 
4 


handheld 
computing 


90s 


„Хо virtual 
reality 
glasses 
(2016) 


UbiComp / pervasive 
computing 
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Introditiion to Human-Computép Interaction satel 


• At physical level, НСІ concerns the selection of the most appropriate 
input devices and output devices for a particular interface or task 


Determine the best style of interaction, such as direct manipulation, 
natural language (speech, written input), WIMP (windows, icons, 
menus, pointers), ВСІ, etc. 


Develop or improve: v = 
Safety by = Bl 
Utility | 
Effectiveness H | 
Efficiency 
Usability | | PF 
Appeal P А IB S 


of systems that include computers in a wide sense 
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Safety: protecting the user from dangerous conditions anc 
sirable situations 


Users: how to prevent user physical security during 
the utilization of the computer. For example, in a 
nuclear energy plant or bomb-disposal, the 
operators should interact with computer-based 
systems remotely. 


Medical equipment in intensive care unit (ICU) are 
remotely managed and consulted. 


Data: how to prevent user from making serious 
errors by reducing risk of wrong keys/buttons being 
mistakenly activated. 
* Provide user with means of recovering errors 
* Ensure privacy (protect personal information such as habits and 
address) and security (protect sensitive information such as 
passwords, VISA card numbers) 
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цу: it is the extent of providing the right kind of functio 
n do what they need or want to do. 


* |n computers, a utility is a small program that provides an addition to the 
capabilities provided by the operating system. In some usages, a utility is 
a special and nonessential part of the operating system. 


* The degree to which objectives are achieved and the extent to 
which targeted problems are solved. In contrast to efficiency, 
effectiveness is determined without reference to costs and, 
whereas efficiency means "doing the thing right," effectiveness 
means "doing the right thing." 
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Efficiency: it is a measure of how quickly users can 
ccomplish their goals or finish their work using the system. 


Consider the scenario: a shopping Website provides all the 
information, instruction and server-side support required to 
perform an on-line purchase. However, the users cannot figure out 
how to find the items they want to buy. How efficient is it? 


The Wikipedia definition: EFFICIENCY is the (often measurable) ability to 
avoid wasting materials, energy, efforts, money, and time in doing 
something or in producing a desired result. In a more general sense, it is 
the ability to do things well, successfully, and without waste 


The concept applied to computing: 

* Algorithmic efficiency: optimizing the speed and memory requirements of 
a computer program 
Storage efficiency: effectiveness of computer data storage 
Efficiency factor: the ratio of the time to transmit a text automatically at a 
specified modulation rate to the time actually required to receive the 
same text at a specified maximum error rate. 
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Usability: how ease of learning and ease of 
ication or service. 


Can | use the basic functions of a new digital camera without reading the manual? 
Does the software facilitate us to learn new functions easily? 


— —"" Usability ISO 9241 definition: "it is 
a measure of the effectiveness, „Бъта weh 
efficiency and satisfaction with Це: па Layout develo post 
which specified users can achieve сарі lu 
specified goals in a particular usability 
environment.” 


Appeal: how well the user likes the 
* First impression 
* Long-term satisfaction 
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USABILITY can be seen as... 
* ап outcome: some applications, the websites (and may be the robots?) 

that are usable. 
a process: itis a methodology or approach (usually called "user- 
centered design") 
a set of techniques: usability testing, contextual inquiry, heuristic 
evaluation – there are many techniques whose aim is to improve 
usability, its implementation and its quantification. 
a philosophy: when improved usability is a value that motivates the 
way in which products are developed 


Usability engineering: a practical and systematic methodology to deliver a product that 
work correctly for the user. It involves several methods: 


Gathering requirements 
Designing 
Цевя-Сактало Developing and testing prototypes 
DESIGN PROCESS $ А 
Evaluate design alternatives and 
analyse usability problems 
Propose solutions and test with the 
users 
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To understand users, we need to involve them! 


Designers’ Users’ 
Conceptual Model Mental Model 
о о 


о о 
о о 
User 
consultation 
—— < 
Nb. FM 
и 


User Interface 
Prototype 
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The user is in the centre: USER-CENTRED design 
Reimagining HCI: 
Toward a More Г 
Human-Centered Perspective Р e / | 


é de Technologie de Troyes | liamjbannon@gmail.com 


3 


Some years ago, НС! researcher Panu Korhonen of Nokia 
outlined to me how HCI is changing, as follows: 


•|п the early days the Nokia НС! people were told "Please This, in a nutshell, te 
evaluate our user inter- face, and make it easy to use." 

That gave way to "Please help us design this user story of how HCI has 
interface so that it is easy to use." That, in turn, led to a moved from evaluation of 
request: "Please help us find what the users really need interfaces through design 
so that we know how to design this user interface." systems and into 

*And now, the engineers are pleading with us: "Look at al sense-making of 
this area of life, and find us something interesting!” 
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User-Centred Development – How is it Different? What is 
different in the process? 


User-centric, not data-centric (involves users in the entire 
process as much as possible). 

Highly interdisciplinary (draws on knowledge from a 
multitude of areas: art, psychology, technical writing, 
computer science... ) 

Highly iterative (involves as much testing and revision as 
possible, especially before final implementation) 


User-centred design cycle: 
Data collection 
Data analysis 
User modelling 
Design 
Prototyping 
Evaluation 
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User-centred design principles 


Develop 


‘or 


Centered 
Design 


Interaction 
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Access to users is not always easy: introducing 'personas' 


Personas 


o № 
` $ 


= 


| “Roles 
б " 
8 | 252; 
uo» 
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Audience 
1 
T 
1 
22 


Ге 
Го 
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"| am looking for technologies that can bring me closer to my children and 


grandchildren..." 


Approach to technology: 


f Hanna has a computer at home with Windows ХР as 
an operating system. 

fe She owns а 2G mobile phone, without camera. 
f'eShe uses Internet to check her emails and for 
Internet banking. 


Barriers: 

/* Mild visual impairment (hyperopia), she 
has trouble seeing clearly up close. 

f She suffers from mild-moderate hand 
osteoarthritis (bone and cartilage 
degeneration). 
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Francesca | "My worst fear is to end up alone in a nursing home..." 


Approach to technology: 

f'*She owns an old computer at home and uses it 
mostly to store pictures about her holidays and to let 
the grandchildren play on it when they come to visit. 
f* She owns а 26 mobile phone, with no extra 
functionality than calling and receiving phone calls. 


Barriers: 

(> Moderate-severe hearing impairment, she 
is using a hearing aid all the time. 

fe She suffers from mild memory loss. 
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Scenarios 


* How would our personas use an IT system? 


* Think of a scenario and work it through 
* |dentify implications 

— design issues 

— possible errors 

— task sequences 

— goal achievement 
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Example scenario 
Title: "Мо Pan On Hot Hob" (NPOHH) 


Persona: Nikos Situation: 
Light visual impairment Nikos is in the bathroom having a bath. He 
№ Deaf switched on a hob but did not put any pan on 
it. The system sends a notification to the user 


and asks him if he wants to use it or if the hob 
should be turned off. 


515 in the bathroom 








Case number: 1 Пай: 14/09/2007 
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Another one 
Title: The Washing Cycle Has Finished (TWCHF) 


Situation: 
Persona: Mikkel 


Mikkel is watching TV when the washing cycle 
- Mild hearing impairment 


is finished. The system sends a notification to 
- Diabetes Mikkel to let him know that he can now 


unload the washing machine. 


E-Servant 


— 


3 


Notification sent to user 
Notification sent by 
broadcast to HMI dient: 


—% 


Mikkel ts watching ТУ with 
a very loud volume He ha: 
the portable controller by 
his side 











Appliance: Washing Machine Case number: 1 Date: 13/09/ 2007 
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Use Cases 510 4 KATY ARE WATCHING THE SOAPS... 
DONT YOU EVER! EVER! 


Second screen entertainment. 
“Dumped by Twitter” 
http://toondoo.com 


* 
VOTE | 


Y 
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Another one... 


SHOPPING 15 SO SIMPLE 
AND FUN THESE DAYS HEHE! 
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Storyboarding/wireframing (e.g. draw.io) 


Gili ehe 184- 97-23 
— * ав) 
белем Ша 
ТУ 

Die frst с. ін М 70 
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Brainstorming (mind mapping techniques) 


A 
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Affinity diagramming 
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The Wizard of Oz technique 


http://www.youtube.com/watch ?v=NZR64EF30pA 
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Wizard of Oz in a lab setting 
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Human-computer interfaces 


The design of effective human-computer interfaces has been, and 
continues to be, a significant challenge that demands an appreciation 
of the entire human perceptual-motor system. 


"Technological advances have undoubtedly served to improve the HCI 
experience. For example, we have progressed beyond the use of 
computer punch cards and command-line interfaces to more complex 
tools such as graphical user interfaces, speech recognition, eye-gaze 
control systems and ВСІ. As НС! has become not only more effective, 
but by the same token more elaborate, the importance of the 
interaction between the various perceptual, cognitive, and motor 
constraints of the human system has come to the forefront." 


Human Computer Interaction fundamentals. A. Sears 2009 
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Interaction 


The Reactable SERGI JORDÀ UPF Musical 
interactive instrument 


— 


— 


> 
зи 
-T — 


https://www.sourceaudio.ne 
t/hot-hand.html 


5 


22) — ра The Hothand 
^ 


— — 
» 


== 


https://www.youtube.com/watch?v=mH7w7R4kiFU 


Examples of new interaction schemes 
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Human information processing 


Human information processing theory deals with how people receive, 

store, integrate, retrieve, and use information 

According to Huitt (2003), there are a few basic principles that most 

cognitive psychologists agree with: 

* The mental system has limited capacities, i.e. bottlenecks in the 
flow and processing of information. 
A control mechanism is required to oversee the encoding, 
transformation, processing, storage, retrieval and utilization of 
information. 
There is a two-way flow of information. Sensory input is combined 
with information stored in memory in order to construct meaning. 
The human organism has been genetically prepared to process and 
organize information in specific ways 
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decay 


Sensory d displacement 
Registers T d 


interference 


SHORTTERM STORE | LONG TERM STORE 
[temporary store | _ permanent store | 


control processes 
rehearsal coding 


https://www.youtube.com/watch?v=7G9IK mUmRE 
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Cognitive Processes 


e Cognitive processes include perception, recognition, 
imagining, remembering, thinking, judging, reasoning, 
problem solving, conceptualizing, and planning. These 
cognitive processes can emerge from human language, 
thought, imagery, and symbols. 

In addition to these specific cognitive processes, many 
cognitive psychologists study language-acquisition, 
altered states of mind and consciousness, visual 
perception, auditory perception, short-term memory, 
long-term memory, storage, retrieval, perceptions of 
thought,... 
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Information-processing framework 


Attention is the collection of processes that allow us to dedicate our 
limited information processing capacity to the purposeful (cognitive) 
manipulation of a subset of available information. 


Stated another way, attention is the process through which 
information enters into working memory and achieves the level of 
consciousness. 


There are three important characteristics of attention: 


(a) attention is selective and allows only a specific subset of information 
to enter the limited processing system; 

(b) the focus of attention can be shifted from one source of information 
to another; 

(c) attention can be divided such that, within certain limitations, one may 
selectively attend to more that one source of information at a time 
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Emotion 


Physiological 
Responses 


Sensory 
Input 


Neurological structure of emotion 

Thalamus: signal processor. The direct thalamic-limbic pathway is the 
mechanism that accounts for the more primitive emotions 

The limbic system, "seat of emotion," constantly evaluates the need/goal 
relevance of its inputs. If relevance is determined, the limbic system sends 
appropriate signals both to the body, coordinating the physiological response, 
and also to the cortex, biasing attention and other cognitive processes 





The limbic system,1 often called the "seat of emotion,” constantly evaluates the 
need/goal relevance of its inputs. If relevance is determined, the limbic system 
sends appropriate signals both to the body, coordinating the physiological response, 
and also to the cortex, biasing attention and other cognitive processes 
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One of the most important effects of emotion lies in its ability to 
capture attention. Emotion-relevant thoughts then tend to dominate 
conscious processing—the more important the situation, the higher 
the arousal, and the more forceful the focus (Clore & Gasper, 2000). 
Emotion's effect on attention also has implications for memory. 
Because emotion focuses thought on the evoking stimulus, emotional 
stimuli are generally remembered better than unemotional events 
(Thorson & Friestad, 1985). 

Mood has also been found to affect cognitive style and performance 


Mood has also been shown to influence judgment and decision 
making. 
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MEASURING AFFECT/EMOTIONS 


Measuring user affect can be valuable both as a 
component of usability testing and as an interface 
technique. When evaluating interfaces, affective 
information provides insight into what a user is feeling— 
the fundamental basis of liking and other sentiments. 


Neurological responses: 


The brain is the most fundamental source of emotion. In 
a relaxed state, the human brain exhibits an alpha 
rhythm, which can be detected by EEG recordings. 
Disruption of this signal (alpha blocking) occurs in 
response to novelty, complexity, and unexpectedness, as 
well as during emotional excitement and anxiety. 
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MEASURING AFFECT/EMOTIONS cont. 


Autonomic Activity 


* Heart rate, blood pressure, blood-pulse volume, 
respiration, temperature, pupil dilation, skin 
conductivity, and muscle tension measured by 
electromyography EMG. Combined measures of 
multiple autonomic signals show promise as 
components of an emotion recognition system. Picard, 
Vyzas, and Healey, for example, achieved 8196 percent 
recognition accuracy on eight emotions through 
combined measures of respiration, blood pressure 
volume, and skin conductance, as well as facial muscle 
tension. 
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A model of Calm HCI. JNA Brown, G Leitner, M Hitz, A Català. 2015 


INTERFACE 
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Three different levels of attentiveness 





Signals designed to trigger only the pre- attentive portion the sensory system in order 
to perceive subtle signals and recognise familiar patterns without interrupting 
attentive focus 
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Affective computing: computing that relates to, 
arises from or deliberately influences emotions 


Founding member of the first IEEE 
Technical Committee оп Wearable 
Information Technology, which was the 
precursor to the  Qquantified-self 
movement that | promotes self- 
monitoring using wearable sensors and 
other technologies. Rosalind Picard. 
MIT Affective Computing Lab MIT 
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MULTIMODAL INTERACTION 


People naturally interact with the world multimodally, through both parallel and 
sequential use of multiple perceptual modalities. Multimodal human-computer 
interaction has sought for decades to endow computers with similar capabilities, in 
order to provide more natural, powerful, and compelling interactive experiences. 
(speech, touch, vision, and gesture) Matthew Turk. Pattern Recognition Letters 


While the multimodal interaction community has focused more on input 
technologies such as speech and gesture recognition and haptic input, and 
multimodal output has been a key element of multimedia and visualization 
research communities, the overall goal of multimodal interaction is to fully 
support both directions of communication between human and machine as well 
as to empower computer-supported human-human multimodal interaction. 
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MULTIMODAL INTERACTION EXAMPLE 


Augmented Reality-based Remote Collaboration 


Steffen Gauglitz, Benjamin Nuernberger, Kuo-Chin Lien, Cha 
Lee Matthew Turk, Tobias Hóllerer 


We describe a framework and several prototype implementations for 
unobtrusive mobile remote collaboration on tasks that involve the 
physical environment. Our system uses the Augmented Reality 
paradigm and model-free, markerless visual tracking to facilitate 
decoupled, live updated views of the environment and world-stabilized 
annotations while supporting a moving camera and unknown, 
unprepared environments. 


https://www.youtube.com/watch?v-tlJUOGi6hf 
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On Pervasive computing, also called ubiquitous computing, is the growing 
trend of embedding computational capability (generally in the form of 
microprocessors) into everyday objects to make them effectively 
communicate and perform useful tasks in a way that minimizes the end 
user's need to interact with computers as computers. Pervasive computing 
devices are network-connected and constantly available. 


> 


< Ч» 


To consider: technology trends, users' profile, 
users' abilities... 
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new ideas for Person-centred design. 


Virtual reality: A computer simulation of a real or 
imaginary system that enables a user to perform 
operations on the simulated system and shows the 
effects in real time. 


Augmented reality is the integration of digital 
information with the user's environmentin real time. 
Unlike virtual reality, which creates a totally artificial 
environment, augmented reality uses the existing 
environment and overlays new information on top of 
it. 


Attentive Interfaces: The computer should attend to 
the user's needs through anticipating what the user 
wants to do; i.e., interfaces respond to the user's 
expressions and gestures 
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COGNITIVE ENGINEERING 


Mathematical 


Product Experimental Model 
Formulation 


SINAPSE. Cognitive rehabilitation 
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field of study dedicated to 


N 


E 4 ћ а ~ 
| | А. ‚ \ understanding, designing, and 
і Yik: 4 Mord DO evaluating robotic systems for use by o 
3 with humans. 


— AT b "A o 
t | | y а 
, "фо 
УУ 
у и» > 


Social Robots 
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On Social aspects of HCI 


E-inclusion is a social movement whose goal is to end the digital divide, a term 
used to describe the fact that the world can be divided into people who do and 
people who don't have access to - and the capability to use - modern 

information technology (IT). According to advocates, e-inclusion has the power 

to: close the gap between developed and less developed countries; promote 
democracy and mutual understanding. - 
Ethical issues: НС! researchers аге now designing and | 

evaluating technologies in increasingly sensitive and 

challenging settings. It is crucial to consider and plan for 

ethical issues when conducting any research involving i 

people, but this becomes even more important when the p IN - 
research is conducted in sensitive settings. Research that 

involves vulnerable or marginalized participants, or that is 

emotionally challenging for researchers, can produce 


complex ethical dilemmas where there are no clear "right" or 
"wrong" answers. 
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Recent НС! significant journal papers 


How Can Affect Be Detected and Represented in Technological Support for Physical 
Rehabilitation? Temitayo A. Olugbade University College London, United Kingdom 2019 
Although clinical best practice suggests that affect awareness could enable more effective 
technological support for physical rehabilitation through personalisation to psychological 
needs, designers need to consider what affective states matter, and how they should be 
tracked and addressed. 


Understanding the Role of Interactive Machine Learning in Movement Interaction Design. 
Marco Gillies.2019. Interaction based on human movement has the potential to become an 
important new paradigm of human-computer interaction. However, high quality, mainstream 
movement interaction requires effective tools and techniques to support designers 

Subjective sleep quality monitoring with the Hypnos digital sleep diary: evaluation of usability 
and user experience T. Vacaretu, N. Batalas, B. Erten - Uyumaz, M. van Gilst, S. Overeem, Panos 
Markopoulos 2019 Sleep diaries are often used to support the diagnosis and treatment of sleep 
disorders though the emergence of self-tracking technologies also makes them interesting for 
intrinsically motivated individuals who wish to gain insight into their sleep patterns and related 
influences. This paper introduces Hypnos, a digital sleep diary, and a user study aimed to its 
usability and the resulting user experience 
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Key Note 


End-User Development for the Internet of Things Panos Marcopoulos TU/e 2109. 

The Internet of Things promises to put within arm's reach a vast network of connected 
computerized devices, from sensors and actuators to more traditional devices, such as phones 
and tablets. Research in this field has been primarily focused on technical considerations, such 
as how devices will connect with other devices and how those connections will be kept 
secure. Relatively, less attention has gone into improving the usability of these constellations 
of devices for people using them who live and act within the dynamic technological habitat 
they provide. We do not yet know how best to help users to harness the potential power of 
these large collections of devices to accomplish their tasks. The Internet of Things is likely to 
transform these tasks by enhancing our physical environment and the objects that surround 
us with sensing and computational capabilities, which, in turn, is likely to result in new 
activities and needs for people. 
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Ша од to Human-Computer Interaction : — 


HCI Journals 


ACM Computing Surveys [SJR] 

ACM interactions [MAS] [SJR] 

ACM Transactions on Computer Human Interaction [SJR] 

ACM Transactions on Interactive Intelligent Systems [SJR] 
Advances in Human-Computer Interaction [SJR] 

Behaviour and Information Technology [SJR] 

Cognitive Computation [SJR] 

Computers in Human Behavior [SJR] [SD] 

Foundations and Trends in Human-Computer Interaction [SJR] 
Human-Computer Interaction [SJR] [WoS] 

IEEE Systems Man and Cybernetics, Part A: Systems and Humans [SJR] 
IEEE Transactions on Affective Computing [SJR] 

IEEE Transactions on Human-Machine Systems [SJR] 

Interacting with Computers [SJR] 

International Journal of Ambient Computing and Intelligence [SJR] 
International Journal of Human-Computer Interaction [SJR] 
International Journal of Mobile Human Computer Interaction [SJR] 
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HCI Journals cont. 


International Journal of UbiComp [MAS] 

International Journal of Virtual Reality [---] 

Journal of Ambient Intelligence and Humanized Computing [SJR] 
Journal of Ambient Intelligence and Smart Environments [SJR] 
Journal of Organizational and End User Computing [SJR] 
Journal of Usability Studies [---] 

Multimodal Technologies and Interaction [---] 

Personal and Ubiquitous Computing [SJR] 

Pervasive and Mobile Computing [SJR] [WoS] 

Proceedings of the ACM on Human-Computer Interaction [---] 
User Modeling and User-Adapted Interaction [SJR] 

Virtual Reality [SJR] 
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A memory game 


• http://www.freegames.ws/games/kidsgames/ 
simon/mysimon.htm 


Some light entertainment... 
• Thorndike’s Law of Effect (1898) 
• A more complex problem 
e An example of insight problem-solving 
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